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ABSTRACT 

A meta-analysis including nine samples of mothers in home 
visiting programs (N=l,600) examined differences in the effects of home 
visitor programs on maternal behavior. Although effect sizes varied, programs 
with more frequent visitation had higher success rates. This dosage effect 
explained all of the variance in effect sizes. Other significant variables 
included number of participants, type of administrative organization, and 
year of study publication. Specifically, larger sample sizes predicted lower 
effect sizes. University programs were more successful than programs 
administered by nonprofits or government agencies. Studies conducted more 
recently had lower effect sizes. However, this last finding reflected the 
occurrence of less frequent visitation in later studies. The correlation 
between frequency of visitation and the year of the study was strong. 
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Abstract. A meta-analysis including nine samples of mothers in home visiting 
programs (N = 1600) examined differences in the effects of home visitor programs on 
maternal behavior. Although effect sizes varied, programs with more frequent visitation 
had higher success rates. This dosage effect explained all of the variance in effect sizes 
(Q = 19.01, df = 1, p_< .0001). Other significant variables included number of 
participants, type of administrative organization, and year of study publication. 
Specifically, larger sample sizes predicted lower effect sizes (Q = 4.61, df = 1, p < .005). 
University programs were more successful than programs administered by nonprofits or 
government agencies (Q = 8.25, df = 1, p < .005). Studies conducted more recently had 
lower effect sizes (Q = 7.96, df = 1, p < .005). However, this last finding reflected the 
occurrence of less frequent visitation in later studies. The correlation between frequency 
of visitation and the year of the study was strong (r = -.86) 
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A Meta-Analysis of Home Visitor Programs: 

Moderators of Improvements in Maternal Behavior 

Early intervention programs for families in poverty were established in the 1960s 
following the Supreme Court desegregation case affirming all children’s right to an 
adequate education (Ramey & Ramey, 1998). Children who lived in poverty were 
determined to be at-risk for school failure due to social problems that often accompany 
poverty, such as parental illiteracy and family isolation (Schorr, 1988). Family support 
programs and other early intervention programs, such as Head Start, attempted to 
encourage cognitive development of young children in poverty. Evaluations of center- 
based and home visiting family support programs followed their development. This 
paper presents a meta-analysis of the home visiting evaluation literature and correlates of 
program effectiveness. 

Two areas of focus among family support programs have been identified: mental 
health prevention and early childhood education (Durlak & Wells, 1997). Programs 
designed to improve children’s mental health are aimed at preventing child abuse, 
children’s mental illness, or juvenile criminal behavior. Educational interventions have 
been designed exclusively to affect academic outcome and are generally reviewed 
separately from programs aimed at mental health prevention. These different models of 
intervention have overlapping effects in two primary early childhood outcomes: socio- 
emotional and cognitive competence (Cowen, 1997). For example, the Perry Preschool 
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Project, which was originally designed to increase academic achivement, showed long- 
term effects of reducing negative socio-political outcomes, such as criminal behavior and 
welfare receipt (Berrueta-Clement, Schweinhart, Barnett, Epstein, & Weikart, 1984). In 
keeping with this finding, home visiting programs may evaluate changes in parenting that 
affect both social-emotional and cognitive development. Improvements in maternal 
sensitivity serve as an indicator of changes in behavior that impact children’s later social- 
emotional skills; improvements in the level of academic stimulation in the home serve as 
an indicator of changes in parental behavior that impact children’s later academic 
aptitude. 

Effects of home visiting programs may vary depending on the characteristics of 
the program. A narrative review of 3 1 home visiting program evaluations suggests that 
the frequency of home visits determines the size of effect (Olds & Kitzman, 1993). We 
reviewed the home visiting evaluation literature using meta-analytic techniques in order 
to quantify the effect of frequency of home visits and to examine other moderators of 
effect size. Three other moderators were of interest: size of program, type of 
administration, and date of study. Theoretically, larger home visitor programs may not 
have concentrated as many resources on individual participants. As programs expand in 
size, the original program direction may change course. Pilot studies run by universities 
in the early years of evaluation research may have achieved superior results because of 
their focus on a smaller group of participants, the quality of home visitor supervision, 
improved training of home visitors, or their high frequency of visitation. 
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Methodology 

A search of the ERIC, Social Work Abstracts, and PsychLit yielded 93 articles. 
The keywords were ‘family support programs,’ or ‘early intervention programs,’ and ‘not 
handicapped.’ An initial search of the literature using keywords 'family support 
programs' or 'early intervention programs' and not 'handicapped' yielded 93 studies. An 
additional 9 studies from our files were also included for inspection. These 102 studies 
were reduced to 7 studies, including 9 different groups of participants (N = 1600). 

Criteria for exclusion from the study were: a pre-test/post-test study design, a center- 
based approach in addition to or instead of a home visitor model, program location 
outside of the United States, and programs for handicapped children. The effect size for 
this initial meta-analysis represents change in maternal behavior, particularly behavior 
that directly impacts the child’s development. 

Twenty-four items were included in the two coding sheets that addressed the 
individual study. These two pages listed study variables from three different categories: 
(1) basic identifying data, such as year of publication and status; (2) characteristics of 
subjects, such as average age of mothers; and (3) intervention characteristics, such as 
frequency of home visits. Regarding the frequency of home visitation, differences may 
arise between the intentional frequency and the actual frequency. In cases where both 
numbers were reported, we used the actual frequency of home visitation. Additional 
coding sheets for each study reported information related to calculation of effect sizes. 

All studies were double-coded; differences were resolved by consensus. 
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Measures 

We identified two types of scales in these studies that directly measure maternal 
behavior. These two categories, which indicate maternal sensitivity or an academically 
stimulating home environment, were used to determine the effect sizes. It should be 
noted that the items within these two types of scales often address similar concepts. For 
example, a subscale of the HOME (Caldwell & Bradley, 1984), emotional and verbal 
responsivity, is basically a measure of sensitivity. Another home environment measure 
specific to the study contains questions about parental involvement (Jester & Guinagh, 
1983). All of the home environment measures reviewed in this paper included some 
elements of maternal sensitivity. 

When home environment and sensitivity measures were both used in the same 
study, we randomly selected one of the measures for inclusion in the data set. To justify 
random selection, we used the Q statistic, a chi-square value used in meta-analysis, to 
analyze differences between the two types of measures. A categorical analysis of effect 
sizes showed no significant difference between effect sizes from sensitivity and home 
environment measures (Qbetween= 1-76, df = 1, p < .25). An examination of the data on an 
individual level indicated similar effect sizes for sensitivity and home environment within 
each of the three studies that used both types of measures. Given this evidence, we used 
both types of effect sizes from these two constructs within the meta-analysis. 

Four measures acted as moderators of improvements in maternal behavior: (a) 
dosage, as defined by the number of visits per year, (b) number of participants, (c) date of 
study publication, and (d) type of administrative organization. The following section will 
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discuss moderating effects of these four variables and the relations among these 
moderators. 

Results 

Nine groups of participants from seven studies included a total of exactly 1600 
participants. Two groups served only teen mothers. The age of child assessment ranged 
from one to four years. For six of the seven studies, the program lasted at least one year. 

Figure 1 shows confidence interval plots of the effect sizes measuring the 
difference in maternal behavior between experimental and control groups. All but one of 
the effect sizes was positive. The study resulting in a negative effect size was somewhat 
different than other home visiting programs in that home visitors worked with clients on 
setting goals and controlling impulsivity, whereas other programs focused on parenting 
skills, case management, and social support. We included this program in the study 
because it met all other study criteria. 

The confidence interval plot suggested that the studies were not homogeneous; 
that is, they did not share a common effect size. A chi-square test for homogeneity was 
significant (Q = 29.58, df = 8, p < .005), confirming that the studies were heterogeneous. 
The mean of the effect size, weighted by sample size, was 0.12, with a 95% confidence 
interval ranging from -.1 to .3. This confidence interval included an effect size of zero. 
However, these statistics are not meaningful, because the Q-statistic indicates that the 
fixed-effect model cannot be used in this case. The studies did not represent a single 
population of outcomes and should not be combined. In such cases. Hedges (1994) 
suggests looking for a moderating variable. Within sets of studies, certain characteristics 
may determine study outcomes. 
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A regression of effect size on home visitation frequency yielded the chi-square 
value of 19.02 (df = 1, e < .005). QResiduai (10.54, df = 7, e < .025) indicated that there 
was no significant residual variance. The variable of home visitation frequency 
explained all of the variance among effect sizes. Thus, the studies may be treated as a 
homogeneous set when home visitation frequency is controlled, and a synthesis of effect 
sizes is meaningful. 

A categorical analysis indicated that effect sizes were more likely to occur when 
visitation frequency exceeded 30 times a year. The weighted mean for this middle 
category of home visitation was .29; the confidence interval (.02 to .56) did not include 
zero. This indicated a positive effect of home visitation on maternal behavior across all 
studies in this category. Effect sizes more than tripled in studies with 60 or more visits a 
year. The weighted mean of this category was equal to 1.14, with a confidence interval 
from .61 to 1.67. 

Another test for significance of home visitation frequency uses the standard error 
of the beta from the regression equation. The standard error of bi is 0.002; therefore, the 
95% confidence interval (.005 to .013) did not include zero. The resulting equation, T = - 
.114 + .0093 (number of visits per year), indicated that it is necessary to have more than 
12 visits per year to achieve an effect size greater than zero (Figure 2). We concluded 
that home visitation does produce a positive effect, dependent on the level of home 
visitation frequency. 

Other variables included in the hypotheses also proved to be significant predictors 
of effect size. Date of study publication (Q = 7.96, df = 1, p < .005), status of 
administrative organization (Q = 8.25, df = 1, e < -005), and size of sample (Q = 4.61, df 
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= 1, 2 < -05) were significant variables when entered individually. However, all of these 
variables left significant unexplained residual variance. These regressions show that 
earlier publications tended to have larger effect sizes. Government-based programs 
tended to have smaller effect sizes when compared with university or private programs. 
Programs administered through universities had the largest effect sizes. Larger programs 
tended to have smaller effect sizes. Adding these moderators to the model did not add 
significant explanation to the univariate model that included only home visitation 
frequency. 

Intercorrelations among the moderator variables indicated that study 
characteristics may be confounded with frequency of visitation. It appears that programs 
evaluated in later studies had expanded in size. The number of participants in later 
studies tended to be larger than the number of participants in earlier studies (r = .42). 
Government funding may have increased the number of participants (r = .66), but the 
frequency of visitation also tended to diminish (r = -.74). In fact, there may be some 
problems with multicollinearity in a model that includes all variables. This finding is 
particularly interesting placed in the historical perspective of the family support 
movement. An analysis of early childhood interventions discussed the differences 
between early model programs and later government implementation (Gomby, Lamer, 
Stevenson, Lewit, & Behrman, 1995). The intensity of early programs may have been 
diluted to meet fiscal constraints accompanying government funding and program 
expansion. Of course, substantive interventions must accompany frequent home 
visitation. 
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Limitations and Directions for Future Research 

Although the our sample provided a much-needed focus on quality home visiting 
evaluation, it did not allow for measurement of the effects of home visitor training level 
or type of population. For six of the seven programs, the only educational qualification 
for home visitors was a high school diploma plus some training that was provided by the 
agency. One program’s home visitors all had at least some college experience (Wagner 
& Clayton, 1999); however, this was not sufficient to examine differences in home 
visitors’ educational levels. 

The comparison of programs with differing levels of home visitor training is an 
interesting question. While higher levels of education are generally accepted as 
preferable, it is possible that higher education of home visitors combined with 
characteristics of families can have a negative effect on the success of programs. For 
example, in the Project Beethoven intervention involving the Robert Taylor housing 
project of Chicago, residents were unable to relate to college-educated home visitors 
(Curtis, 1995). When residents from the project were recruited for work as home visitors, 
outcomes of the program improved. 

A few studies have indicated costs and benefits involved in home visiting 
programs (Olds & Kitzman, 1993). Others have listed only cost per family. The articles 
reviewed here did not indicate average cost; however, a cost variable in combination with 
other data from a broader range of studies may assist in the design of future home visitor 
programs. A meta-analysis including specific program characteristics, such as the level of 
structure within the program or the type of theoretical basis for the program, may also 
provide information that would be useful to practitioners. A variety of sample 
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characteristics may indicate that certain demographic groups benefit from family support 
programs than other groups. These questions have been debated in qualitative literature 
reviews (St. Pierre & Layzer, 1998; Yoshikawa, 1995). 

In addition to future meta-analytic research, additional primary research is called 
for in engagement and retention of families. Our findings clearly indicate the importance 
of regular and frequent home visiting. Importantly, family support programs do appear to 
produce improvements in maternal behavior, given frequent and regular visitation by the 
home visitor. 
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CONFIDENCE INTERVAL PLOT: 
MATERNAL BEHAVIOR EFFECTS 
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Figure 1 . Confidence interval plot. 

This figure suggests that effect sizes are heterogeneous. 
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VISITS PER YR 

Figure 2. Visits per year vs. effect size. 

This indicates that intensity of home visitation has a positive effect on program outcome. 
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